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1.0  Summary 

The  focus  of  this  study  was  the  clinical  impact  of  ear/nose/throat  (ENT)  consultations 
conducted  via  video  teleconferencing  (VTC)  during  a  planned,  4-month  study  period. 
This  study  included  structured  interviews  with  TRICARE  Region  9  physicians  and 
nonphysician  medical  personnel.  The  interview  information  was  used  to  compare  and 
contrast  the  attitudes  of  users  and  measure  their  recent  use  of  telemedicine. 

1.1  Background 

This  study  was  a  follow-up  to  a  previous  retrospective  evaluation  of  the 
telemedicine  network  in  TRICARE  Region  9.  This  research  responds  to  a  specific 
naval  directive  to  study  the  potential  of  telemedicine  technologies  such  as  the 
Internet  and  live  VTC  for  shipboard  medical  departments.  Since  limited 
opportunity  exists  to  study  telemedicine  use  at  sea,  the  aim  of  this  research  was  to 
evaluate  current  telemedicine  use  in  remote  military  treatment  facilities  (MTFs) 
ashore  with  limited  access  to  specialty  care.  The  results  will  be  useful  to  guide 
future  telemedicine  applications  for  different  types  of  ships  and  medical 
providers. 

1 .2  Method 

This  study  was  a  prospective  evaluation  of  ENT  consultations  that  were 
conducted  by  an  ENT  specialty  physician  via  VTC.  These  VTC  sessions  were 
conducted  by  the  specialist  from  the  Naval  Medical  Center,  San  Diego  (NMCSD) 
for  patients/providers  at  remote  MTFs  in  TRICARE  Region  9  and  Lemoore  Naval 
Air  Station  (NAS),  Lemoore,  CA.  Observations  were  recorded  during  a  planned 
4-month  study  period  (November  1, 2000,  to  March  1, 2001).  All  consultations 
were  scheduled  and  recorded  using  the  TRICARE  Region  9  Internet  Web  site,  the 
Referral  Management  System  (RMS). 

Data  sources  included  the  RMS  and  structured  interviews  with  physician  and 
nonphysician  medical  personnel  in  the  TRICARE  Region  9  telemedicine  network. 
The  RMS  captured  data  on  various  factors  related  to  telemedicine  use  (e.g., 
patient  history,  MTF)  and  clinical  impact  (provider  and  consultant  diagnosis).  The 
interview  questions  assessed  general  attitudes  and  recent  experiences  with 
telemedicine. 

1 .3  Results 

A  total  of  193  ENT  consultations  were  conducted  via  VTC  following  referrals 
from  primary  providers.  The  patients  were  mostly  young,  active-duty  military 
personnel.  Of  these  consultations,  45%  led  to  a  change  in  diagnosis  by  an  ENT 
specialty  physician  relative  to  initial  diagnosis  by  the  referring  provider.  This  rate 
of  clinical  impact  generalized  across  different  ENT  conditions  (tonsil,  upper 
respiratory,  hearing),  age,  gender,  military  status  (e.g.,  active  duty  or  dependent) 
and  MTFs. 
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Medical  personnel  reported  generally  positive  attitudes  on  telemedicine 
technologies  and  the  telemedicine  process  in  TRICARE  Region  9.  The 
nonphysicians  reported  slightly  more  favorable  attitudes  and  used  telemedicine 
more  often  and  in  more  ways  than  the  physicians.  Medical  personnel  noted  some 
specific  limitations  in  satisfaction  with  technologies  (e.g.,  VTC  sound),  the 
telemedicine  process,  and  the  RMS. 

1.4  Conclusions 

The  ENT  consultations  via  VTC  had  substantial  clinical  impact.  This  was  true  for 
a  variety  of  ENT  conditions  among  military  personnel  and  dependents  at  remote 
MTFs  where  patients  had  limited  access  to  this  medical  specialty.  These  results 
replicated  and  extended  the  findings  of  a  recent  retrospective  study  of  the 
TWCARE  Region  9  telemedicine  network. 

The  medical  personnel  noted  that  the  RMS  functioned  well  in  coordinating  the 
telemedicine  consultations  for  ENT.  They  were  generally  satisfied  with  the 
telemedicine  technology  and  process.  Compared  with  physicians,  nonphysicians 
made  more  extensive  use  of  telemedicine  and  reported  slightly  more  favorable 
attitudes  toward  satisfaction  with  telemedicine  than  the  physicians.  The  specific 
limitations  of  telemedicine  are  discussed  along  with  recommendations  for  further 
work. 
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2.0  Introduction 


Increasingly,  it  has  become  clear  that  the  effective  application  of  telemedicine 
technologies  such  as  email,  the  Internet,  and  live  VTC  is  essential  for  the  U.S.  armed 
forces  to  optimize  their  healthcare  mission  (Patel,  1994;  Stoloff,  Garcia,  Thomason,  & 
Shia,  1998;  Vidmar,  1999;  Walters,  1996).  This  is  especially  true  for  personnel  deployed 
or  stationed  in  remote  environments  (e.g.,  Cubano  et  al.,  1999;  Nice,  1987).  Telemedicine 
can  benefit  the  Navy  by  providing  timely  access  to  specialized  care  for  patients  and 
medical  personnel  on  ships  at  sea  with  limited  resources  (Nice,  1987).  Telemedicine  use 
provides  many  benefits  (Bashshur,  Reardon  &  Shannon,  2000;  Jerome  et  al.,  2000;  Sinha, 
2000),  and  these  benefits  extend  to  personnel  and  their  dependents  stationed  at  MTFs  in 
medically  underserved  regions  ashore  (e.g.,  HRSA,  1997;  Melcer,  Crann,  Hunsaker, 
Deniston,  &  Caola,  2001). 

This  research  responds  to  a  specific  Navy  research  directive  to  determine  how 
telemedicine  technologies  could  be  used  by  shipboard  medical  departments  of  various 
sizes  (Patel,  1994).  Larger  ships,  such  as  carriers  and  amphibious  ships,  have  extensive 
medical  departments  including  physician  and  nonphysician  personnel  to  perform  the 
functions  of  a  small  hospital  (e.g.,  surgery).  Some  ships  currently  have  telemedicine 
technologies  installed,  such  as  the  Internet,  VTC,  and  computed  radiology. 

Unfortunately,  few  evaluation  studies  of  the  shipboard  telemedicine  process  exist,  and 
they  are  limited  to  these  large  ships  (e.g.,  Larson,  Burr,  Pearsall,  &  Silva,  1998). 
Moreover,  no  telemedicine  capabilities  currently  exist  on  small  ships,  such  as  cruisers 
and  destroyers,  to  assist  limited  medical  departments  headed  by  independent  duty 
corpsmen  (IDCs)  (Nice,  1987).  Thus,  it  is  useful  to  study  telemedicine  where  it  is 
currently  used  at  MTFs  of  various  sizes  ashore.  The  field  of  telemedicine  in  general 
needs  more  systematic  evaluation  studies  (Brender,  Nohr,  &  McNair,  2000). 

The  present  study  was  a  follow-up  to  a  retrospective  study  of  the  first  2.5  years  of 
telemedicine  network  operation  in  TRICARE  Region  9  (Melcer  et  al.,  2001).  In 
TRICARE  Region  9,  remote  MTFs,  such  as  Fort  Irwin  and  Port  Hueneme,  access 
specialty  care  by  physicians  at  NMCSD  via  live  VTC.  Telemedicine  consultations  are 
managed  and  recorded  using  the  RMS,  an  Internet  Web  site. 

The  retrospective  study  showed  that  telemedicine  use  increased  with  the  age  (time  since 
operational)  of  telemedicine  systems  at  various  MTFs  or  the  age  of  individual 
telemedicine  specialties  (e.g.,  ENT).  Most  important,  the  retrospective  study  concluded 
that  approximately  half  of  ENT  consultations  had  a  clinical  impact,  measured  by  change 
in  diagnosis. 

The  present  study  replicated  the  basic  measures  used  in  the  previous  work  to  test  the 
reliability  of  the  findings.  The  present  study  also  extended  the  previous  work  in  two 
important  ways: 

•  Analysis  focused  on  the  ENT  specialty,  the  most  active  telemedicine  specialty  in 
TRICARE  Region  9,  over  a  planned,  4-month  study  period.  ENT  conditions  can 
pose  a  concern  for  shipboard  medical  departments  (Blood  &  Griffith,  1990; 
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D’Alessandro,  D’Alessandro,  Hendrix,  &  Bakalar,  1999;  Derderian  &  Blood, 
1998),  and  little  evaluation  exists  on  the  use  of  VTC  for  remote  ENT 
consultations  in  MTFs  (e.g.,  Blakeslee,  Grist,  Stachura  &  Blakeslee,  1998; 
Sclafani  et  al.,  1999).  The  generality  of  these  VTC  procedures  for  different  ENT 
problems  and  patient  populations  is  unclear. 

•  Interviews  with  military  medical  personnel  (e.g.,  general  medical  officers 
(GMOs),  IDCs,  and  medical  technicians)  were  conducted  to  obtain  quantitative 
assessments  of  their  attitudes  and  experiences  with  TRICARE  Region  9 
telemedicine.  Telemedicine  attitudes  (e.g.,  satisfaction)  and  experience  are 
associated  with  telemedicine  use  (Karp  et  al.,  2000).  Satisfaction  with 
telemedicine  and  use  of  telemedicine  were  compared  between  physician  and 
nonphysician  medical  personnel. 

3.0  Method 

This  was  a  prospective  evaluation  of  telemedicine  consultations  that  were  conducted  by 
an  ENT  specialty  physician.  The  data  are  based  on  information  extracted  from  the  RMS 
database  and  from  interviews  with  physician  and  nonphysician  medical  personnel.  The 
study  evaluates  VTC  consultations  conducted  over  a  planned  4-month  study  period  for 
patients/providers  at  remote  MTFs  in  TRICARE  Region  9  and  Lemoore  NAS. 

3.1  Data  Sources  and  Subjects 

Observations  came  from  two  sources: 

•  A  database  of  telemedicine  consultations 

•  Interviews  with  medical  personnel 

All  personal  identifying  information  (name,  social  security  number)  was  excluded 
from  the  resulting  database  prior  to  analysis. 

3.1.1  RMS  Database 

The  telemedicine  consultations  (N  =  193)  occurred  between  November  1,  2000, 
and  March  1,  2001,  and  were  extracted  from  the  RMS  database  at  NMCSD.  These 
were  all  VTC  sessions  in  which  the  primary  provider  referred  a  patient  for 
consultation  to  an  ENT  specialist  for  initial  evaluation.  Postoperative  and 
follow-up  sessions  were  excluded  from  analysis,  because  these  sessions  usually 
do  not  function  to  determine  diagnosis.  These  consultations  were  provided  for 
patients  treated  at  MTFs  in  TRICARE  Region  9  and  the  MTF  at  Lemoore  NAS. 
These  facilities  comprise  the  functional  telemedicine  network  centered  at 
NMCSD.  Individual  patients  contributed  1  or  more  observations  for  1  or  more 
medical  conditions. 

3.1.2  Medical  Personnel 

A  small  sample  of  medical  personnel  from  the  TRICARE  Region  9  telemedicine 
network  completed  telephone  interviews.  These  were  physicians  (N  =  7)  and 
nonphysicians  {N  =  7),  10  males  and  4  females.  The  physicians  consisted  of  2 
GMOs,  3  primary  care  physicians  and  2  specialty  physicians  (child  psychiatry  and 
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neurology).  The  nonphysicians  consisted  of  1 IDC,  4  medical  technicians,  1 
nurse,  and  1  physician’s  assistant.  All  were  in  group  practices  ranging  from  6  to 
60  providers,  with  a  median  size  of  8.  Of  the  14, 12  were  on  active  duty  and  2 
were  civilians.  They  were  chosen  based  on  referral  by  the  telemedicine 
coordinator  in  TRICARE  Region  9  and  their  availability  for  interview.  At  least  1 
and  as  many  as  4  medical  personnel  worked  in  each  of  8  MTFs.  The  TRICARE 
Region  9  telemedicine  network  is  detailed  in  Table  1. 

3.2  TRICARE  Region  9  Telemedicine  Network 

TRICARE  provides  healthcare  for  active-duty  personnel,  their  qualified  family 
members,  Civilian  Health  and  Medical  Program  of  the  Uniformed  Services- 
eligible  retirees,  and  the  family  and  survivors  of  all  uniformed  services. 

The  lead  agent  for  TRICARE  Region  9  is  located  at  NMCSD,  along  with  the 
resources  for  specialized  care  (i.e.,  the  consultant  physicians).  The  telemedicine 
network  in  southern  California  includes  8  MTFs,  most  with  limited  medical 
specialization.  Table  1  lists  the  MTFs  in  this  network  along  with  their  start  dates 
for  telemedicine  operations. 


Table  1 


MTFs  in  the  TRICARE  Region  9  Telemedicine  Network 


MTF 

Network  Designation 
(distance  from  hub  facility,  miles) 

start  Date 

NMCSD 

Hub  (0) 

6/1/97 

Port  Hueneme 

Spoke  (182) 

8/14/97 

Fort  Irwin® 

Spoke  (213) 

10/17/97 

Edwards  AFB® 

Spoke  (201 ) 

3/26/98 

Vandenberg  AFB*’ 

Spoke  (290) 

11/12/98 

Twentynine  Palms 

Spoke  (174) 

1 1/23/98 

Los  Angeles  AFB 

Spoke  (121) 

3/10/99 

Lemoore  NAS 

Spoke  (327) 

9/16/99 

Note.  MTFs  =  military  treatment  facilities. 

®The  ENT  specialist  was  not  certified  during  part  or  all  of  this  study. 
h"his  facility  was  inoperative  for  6  months  due  to  technical  difficulties. 

NMCSD  functions  as  the  “hub”  of  the  telemedicine  network.  The  remote  MTFs 
are  the  “spokes”  at  various  distances  from  NMCSD.  While  the  Lemoore  facility  is 
not  part  of  TRICARE  Region  9,  it  is  included  in  the  present  study  because  it  is 
part  of  the  functional  telemedicine  network  centered  at  NMCSD. 

Generally,  the  spoke  MTFs  seek  telemedicine  consultations  from  medical 
specialists  at  the  hub  MTF.  This  is  true  for  ENT  (start  date:  June  1997)  and 
neurology  (start  date:  February  1999).  Psychiatry  and  rheumatology  specialists 
are  located  at  Twentynine  Palms  (start  date:  May  1999)  and  Fort  Irwin  (start  date: 
May  1999),  respectively.  Such  a  “hub-and-spoke”  telemedicine  network  model 


5 


has  been  applied  frequently  in  recent  years  in  many  settings  (Jerome  et  al.,  2000; 
Nickelson,  1998). 

3.3  Referral  Management  System 

In  TRICARE  Region  9,  telemedicine  consultations  are  managed  and  recorded  via 
the  RMS,  an  Internet  Web  site.  For  example,  when  a  primary  provider  determines 
a  patient  requires  a  consultation,  the  patient  is  referred  to  a  telemedicine 
coordinator.  If  the  required  specialty  care  is  not  available  at  the  patient’s  MTF, 
the  telemedicine  coordinator  (e.g.,  IDC  or  medical  technician)  will  access  the 
RMS  to  schedule  a  telemedicine  consultation  with  a  specialty  physician,  usually 
located  at  NMCSD.  This  determination  is  based  on  the  availability  of  specialty 
care  at  individual  MTFs  and  the  availability  of  telemedicine  consultants  for  the 
specialty. 

The  patient  at  the  remote  MTF  is  registered  in  the  RMS  with  relevant  personal 
background  information,  reported  symptoms,  and  primary  provider  diagnosis.  The 
RMS  then  allows  the  provider  to  make  an  appointment  with  a  specialty  physician 
at  NMCSD  who  can  access  the  patient’s  information  for  review. 

The  consulting  physician  is  able  to  access  his  or  her  schedule  of  referred  patients 
via  the  RMS.  During  the  consultation  itself,  the  telemedicine  coordinator  presents 
the  patient  to  the  consultant  via  telecommunication  (usually  VTC).  Following  the 
telemedicine  appointment,  the  consultant  enters  comments  on  diagnosis  and 
treatment  in  the  patient’s  RMS  record. 

This  study  focused  on  telemedicine  patients  referred  for  initial  evaluations  by 
ENT  specialists  via  VTC.  The  ENT  specialty  was  chosen  for  analysis  because  it  is 
the  most  active  in  TRICARE  Region  9;  initial  evaluations  via  VTC  were  chosen 
because  these  are  the  sessions  in  which  the  primary  provider  and  specialist 
interact  to  determine  clinical  outcomes  for  patient  care  (e.g.,  diagnosis).  Thus, 
store-and-forward,  preoperative  or  postoperative,  or  follow-up  sessions  were 
excluded,  since  these  sessions  usually  do  not  function  to  determine  diagnosis. 

The  following  variables  were  recorded  from  the  RMS: 

•  Patient  demographics  (age,  gender,  and  military  status) 

•  Reason  for  consultation  (initial,  preoperative, 
postoperative,  follow-up,  other) 

•  MTF  where  the  patient  received  the  telemedicine  consultation 

•  Consultant  medical  specialization  (ENT,  neurology,  psychiatry, 
rheumatology) 

•  Primary  provider  and  consultant  diagnosis 

•  Telemedicine  modality  (VTC  or  store-and-forward) 

The  ENT  consultations  included  a  diversity  of  problems.  All  cases  were  assigned 
to  one  of  the  following  five  major  categories:  hearing  (e.g.,  tinnitus),  sleep-related 
(e.g.,  sleep  apnea),  tonsil-related,  upper  respiratory  (e.g.,  sinusitis),  and 
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Integumentary  (e.g.,  skin  cyst).  These  categories  were  chosen  based  on  discussion 
with  the  lead  ENT  specialist.  Thus,  it  is  possible  to  see  the  relative  frequency  of 
major  ENT  conditions  in  this  population  and  determine  whether  the  clinical 
impact  of  VTC  consultations  varies  among  these  types  of  cases. 

3.4  Interviews  With  Medical  Personnel 

The  medical  personnel  were  asked  to  complete  brief  telephone  interviews, 
approximately  10  minutes  in  length,  to  determine  their  medical  backgrounds  and 
experience  with  technology,  and  to  assess  their  attitudes  toward  telemedicine. 
These  interviews  were  conducted  at  the  end  of  the  study  period  and  followed  a 
structured  survey  questionnaire  format.  The  questions  appearing  on  the 
questionnaire  and  interview  forms  are  based  on  those  used  in  previous  work  at 
Naval  Health  Research  Center  (NHRC)  (Larson  et  al.,  1998).  The  format  was 
influenced  by  Dillman’s  (1978)  methodology  for  mail  questionnaires. 

The  following  variables  were  recorded  from  the  interview  forms: 

•  Medical  positions  (e.g.,  IDC,  nurse,  physician)  and  any  specialization 
(e.g.,  dermatology,  psychiatry) 

•  Years  of  past  experience  with  various  technologies  (e.g.,  computer, 
modem,  Internet,  email) 

•  Medical  purpose  and  years  of  past  experience  with  telemedicine 
(e.g.,  telephone,  RMS,  fax,  VTC) 

•  Attitudes  toward  telemedicine  were  assessed  using  a  5-point  Likert  rating 
scale  (e.g.,  1  =  very  satisfied,  2  =  somewhat  satisfied,  3  =  undecided,  4  = 
somewhat  dissatisfied,  5  =  very  dissatisfied).  5  to  7  questions  were  used  to 
assess  each  of  4  different  attitude  variables: 

-  Satisfaction  with  technologies  (e.g.,  VTC,  Internet) 

-  Satisfaction  with  modalities  (e.g.,  sound,  images) 

-  Usefulness  of  technologies 

-  Perception  of  telemedicine 

3.5  Data  Analysis 

The  data  analysis  was  designed  to  address  the  following  questions: 

•  What  percentage  of  telemedicine  consultations  had  an  impact  on  clinical 
outcomes,  namely  a  difference  in  diagnosis  between  the  primary  provider 
and  specialty  consultant? 

•  What  variables  were  associated  with  changes  in  diagnosis? 

•  What  were  the  attitudes  and  experiences  of  medical  personnel  regarding 
telemedicine?  Were  these  attitudes  and  experiences  related  to  the  use  or 
clinical  impact  of  the  telemedicine  consultations  in  this  study?  Were 
attitudes  related  to  the  extent  of  medical  training  received  by  physician  or 
nonphysician  medical  personnel? 
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A  chi-square  test  (p  <  .05)  was  used  to  test  the  association  between  ENT  problem 
and  change  in  diagnosis.  Descriptive  statistics  are  presented  for  the  interview 
data.  Preliminary  trends  are  described  for  the  interview  data  because  of  the  small 
samples  of  7  physicians  and  7  nonphysician  medical  personnel.  Thus,  the 
comparisons  between  physicians  and  nonphysicians  should  be  interpreted  with 
caution. 

4.0  Results 

A  total  of  193  ENT  consultations  were  conducted  via  VTC  following  referrals  from 
primary  providers.  Forty-five  percent  of  these  consultations  led  to  a  change  in  diagnosis 
by  an  ENT  specialty  physician  relative  to  the  initial  diagnosis  by  the  referring  provider. 
This  rate  of  clinical  impact  generalized  across  different  ENT  problems,  age,  gender, 
military  status,  and  MTFs. 

Medical  personnel  reported  generally  positive  attitudes  on  telemedicine  technologies  and 
the  telemedicine  process  in  TRICARE  Region  9.  Compared  with  physicians, 
nonphysicians  reported  slightly  more  favorable  attitudes  and  used  telemedicine  more 
often  and  in  more  ways. 

4.1  Demographics 

The  sample  consisted  predominantly  of  active-duty  adult  males  between  18  and 
44  years  old  (Table  2).  The  sample  did  include  females  and  military  dependents 
each  representing  approximately  30%  of  the  total  sample  (Table  2). 
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Table  2 


Patients  Referred  for  VTC  ENT  Consultations 


1  Total  Consults 

N 

%  I 

193 

100  1 

1  Gender 

Male 

133 

68.9 

Female 

58 

30.1 

Unknown 

2 

1.0 

1  Age,  years 

1-10 

19 

9.8 

11-17 

5 

2.6 

18-24 

50 

25.9 

25-44 

92 

47.7 

45-65 

19 

9.8 

65+ 

7 

3.6 

Unknown 

1 

.5 

Military  status 

1  Active  duty 

132 

68.4 

Dependent 

59 

30.6 

Unknown 

2 

1.0 

“I? _ 1 

Lemoore  NAS 

110 

57.0 

1  Port  Hueneme 

48 

24.9 

1  Twenty  nine  Palms 

17 

8.8 

I  Fort  Irwin 

15 

7.8 

1  Edwards  AFB 

3 

1.6 

Note.  ENT  =  ear/nose/throat;  MTF  =  military  treatment  facility;  VTC  =  video 
teleconferencing. 

Over  75%  of  the  consultations  came  from  the  Lemoore  or  Port  Hueneme 
facilities.  A  delay  in  the  certification  of  the  ENT  specialist  by  Fort  Irwin  and 
Edwards  AFB  limited  telemedicine  activity  at  these  MTFs. 

These  variables  will  be  considered  further  because  they  may  influence  the  clinical 
impact  of  the  ENT  consultations. 

4.2  Clinical  Impact 

The  telemedicine  consultations  had  a  substantial  clinical  impact  in  the  present 
sample  of  patients.  Almost  half  (45%)  of  the  consultations  produced  a  change  in 
patient  diagnosis  by  the  ENT  specialist  relative  to  the  initial  diagnosis  of  the 
primary  provider  (Figure  1). 
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Figure  1.  Clinical  impact  of  VTC  consultations. 

The  relatively  high  rate  at  which  the  ENT  consultations  had  an  impact  on  patient 
diagnosis  provides  strong  support  for  the  utility  of  this  telemedicine  specialty. 

Since  the  ENT  cases  included  diverse  medical  problems  (e.g.,  hearing-  and  sleep- 
related),  this  study  also  explored  whether  the  overall  rate  of  clinical  impact 
(Figure  1)  was  general  or  limited  to  certain  types  of  ENT  problems,  demographic 
(e.g.,  age,  gender),  or  institutional  variables  (e.g.,  military  or  dependent  status). 
Table  3  shows  the  clinical  impact  as  a  function  of  these  secondary  variables. 
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Table  3 


Clinical  Impact  of  VTC  Consultations  by  Secondary  Variables 


Diagnosis  Changed 

No  Change 

N 

% 

N 

% 

Overall  summary 

87 

45 

106 

55 

Type  of  ENT  problem 


Hearing/vertigo 

23 

46 

27 

54 

Sleep 

12 

38 

20 

62 

Tonsil 

13 

46 

15 

54 

Upper  respiratory 

39 

50 

39 

Integumentary 

0 

0 

5 

100 

Gender 

Male 

63 

47 

70 

53 

Female 

23 

40 

35 

60 

Age,  years 

1-10 

6 

32 

13 

68 

11-17 

2 

40 

3 

60 

18-24 

22 

44 

28 

56 

25-44 

43 

47 

49 

53 

45-64 

9 

47 

10 

53 

65+ 

5 

71 

2 

29 

— 

Military  status 


Active  duty 

62 

47 

70 

53 

Dependent 

23 

39 

36 

61 

MTF 

Lemoore  NAS 

48 

44 

62 

56 

Port  Hueneme 

26 

54 

22 

46 

Twenty  nine  Palms 

5 

29 

12 

71 

Fort  Irwin 

6 

40 

9 

60 

Edwards  AFB 

2 

67 

1 

33 

Note.  Totals  less  than  193  cases  reflect  1  or  2  missing  cases  for  certain  variables 
(e.g.,  192  for  age  and  gender,  191  for  military  status).  VTC  =  video 
teleconferencing. 
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Table  3  indicates  that  the  clinical  impact  of  the  ENT  consultations  generalized 
across  ENT  problems,  age,  gender,  duty  status,  and  MTFs.  No  significant 
deviations  occurred  among  the  major  ENT  categories;  none  of  the  5 
integumentary  cases  led  to  changed  diagnosis,  but  larger  samples  would  be 
needed  to  determine  the  reliability  of  this  effect.  Similarly,  no  major  deviations 
were  seen  within  age,  gender,  military  status,  or  MTF. 

It  is  worth  emphasizing  that  the  active-duty  cases  and  those  18-44  years  old 
showed  rates  of  clinical  impact  similar  to  those  found  in  the  overall  sample.  The 
lack  of  substantial  variation  within  these  4  secondary  variables  supports  the 
generality  of  the  overall  rate  of  clinical  impact. 

4.3  Interviews  With  Medical  Personnel 

Structured  interviews  of  7  physicians  and  7  nonphysician  medical  personnel  from 
the  TRICARE  Region  9  telemedicine  network  supplemented  the  RMS  data  on 
patient  outcomes. 

4.3.1  Past  Telemedicine  Use 

Table  4  shows  the  medical  reasons  for  which  the  physician  and  nonphysician 
medical  personnel  said  they  had  ever  used  telemedicine  technologies.  The 
primary  reason  given  for  past  telemedicine  use  was  to  confirm  diagnosis.  All  but 
1  of  the  14  medical  personnel  had  at  some  point  used  each  of  the  telemedicine 
technologies  to  confirm  diagnosis. 

The  physicians  and  nonphysicians  differed  somewhat  in  past  use  of  telemedicine 
for  educational  reasons  (patient  or  provider).  The  nonphysicians  were  consistently 
more  likely  than  physicians  to  use  telemedicine  for  educational  reasons.  At  least 
70%  of  both  groups  had  used  the  Internet  for  medical  education. 

Table  4 


Past  Use  of  Telemedicine  Technologies  by  Medical  Personnel  (N  =  14) 


Physicians 

Nonphysicians 

Telemedicine 

technology 

Diagnosis 

confirmation 

% 

Education 

% 

Diagnosis 

confirmation 

% 

Education 

% 

Telephone 

100 

43 

100 

76 

Fax 

86 

10 

100 

40 

Email 

100 

48 

100 

81 

Internet 

100 

71 

100 

85 

VTC 

100 

42 

100 

62 

Overall  average 

97 

43 

100 

69 

Note.  VTC  =  video  teleconferencing. 
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4.3.2  Current  Access  to  Telemedicine  Technologies 

Table  5  shows  that  all  nonphysicians  and  most  physicians  in  TRICARE  Region  9 
had  current  access  to  various  telemedicine  technologies.  All  personnel  reported 
well  over  1  year  of  experience  with  each  technology.  Physicians  and 
nonphysicians  reported  similar  technology  experience. 

Table  5 


Current  Access  to  Telemedicine  Technologies  by  Medical  Personnel 


Physicians  Nonphysicians  | 

Technology 

Current  access 

% 

Years  used 

m 

Current  access 

% 

Years  used 
(M) 

Email 

71 

5.1 

100 

3.0 

RMS 

71 

1.4 

100 

1.7 

Internet 

71 

3.1 

100 

3.6 

VTC 

71 

1.7 

100 

1.7 

Note.  RMS  =  Referral  Management  System;  VTC  =  video  teleconferencing. 


4.3.3  Purposes  of  Telemedicine  Use 

Table  6  shows  the  reasons  physicians  and  nonphysicians  used  telemedicine  during 
the  present  study  period  (last  4  months). 

More  than  two  thirds  of  all  telemedicine  activity  was  initiated  to  confirm 
diagnosis.  This  was  true  for  both  groups.  However,  differences  between 
physicians  and  nonphysicians  were  visible  in  the  overall  distribution  of 
telemedicine  purposes.  The  physicians  were  more  likely  than  the  nonphysicians  to 
use  telemedicine  for  provider  education  or  technical  support.  In  contrast,  the 
nonphysicians  were  more  likely  to  use  telemedicine  for  patient  education. 

Table  6 


Recent  Telemedicine  Usage  by  Medical  Purpose 


Confirm 

diagnosis 

% 

Patient 

education 

% 

Provide 

education 

% 

Technical 

support 

% 

Unknown 

% 

Physician 

68 

3 

9 

9 

11 

Nonphysician 

77 

16 

1 

1 

5 

4.3.4  Telemedicine  Activity  and  Satisfaction 

Table  7  summarizes  medical  personnel  reports  of  overall  patient  load  and 
telemedicine  activity  during  the  present  study  period  (last  4  months).  Table  8 
shows  scores  for  medical  personnel  attitudes  toward  satisfaction,  usefulness,  and 
overall  perception  of  telemedicine  technologies  (e.g.,  email,  VTC)  and 
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satisfaction  with  different  modalities  (images,  sound,  written  text)  transmitted  by 
telecommunications. 


Table  7 


Telemedicine  Activity  Reported  by  Medical  Personnel 


Telemedicine  activity 
(last  4  months) 

Physicians 

(Mdn) 

Nonphysicians 

(Mdn) 

1  Telemedicine  patients  seen  (TMED) 

25 

85 

1  Total  patients  seen  (TMED  +  non-TMED) 

900 

200 

1  Telemedicine  ratio  (TMED/TMED  + 

3% 

43% 

I  non-TMED) 

4.3.4.1  Level  of  Telemedicine  Activity 

Table  7  indicates  that  nonphysicians  reported  substantially  more  telemedicine 
activity  than  physicians.  This  effect  is  seen  both  in  the  number  of  telemedicine 
cases  and  in  the  ratio  of  telemedicine  cases  to  all  patient  care  during  the  study 
period  (telemedicine  cases/telemedicine  and  non-telemedicine  cases).  Substantial 
variability  occurred  among  both  physicians  and  nonphysicians  on  these  measures 
and  the  sample  sizes  were  relatively  small  (A''  =  7).  To  reduce  the  effect  of 
extremely  high  or  low  scores,  median  scores  are  presented  for  telemedicine 
activity. 

4.3.4.2  Satisfaction  With  Telemedicine 

Table  8  indicates  that  all  personnel  reported  consistently  favorable  attitudes 
toward  telemedicine  across  all  types  of  questions.  They  rated  their  attitudes  on  a 
5-point  scale,  with  more  positive  attitudes  indicated  by  lower  scores  (e.g.,  1  = 
very  satisfied,  2  =  somewhat  satisfied,  3  =  undecided,  4  =  somewhat  dissatisfied, 
5  =  very  dissatisfied).  Table  8  also  indicates  that  the  nonphysicians  reported 
slightly  more  favorable  attitudes  than  the  physicians,  as  shown  by  mean  scores. 
Although  the  size  of  this  trend  was  small,  it  was  seen  for  each  of  the  4  attitude 
variables  (satisfaction  with  technologies,  satisfaction  with  modality  quality, 
usefulness,  and  perception). 


Table  8 


Telemedicine  Attitude  Scores  Reported  by  Medical  Personnel 


Telemedicine  attitudes 

Physicians 

(M) 

Nonphysicians 

(/W) 

Satisfaction  with  telemedicine  technologies 

1.9 

1.7 

Satisfaction  with  modality  quality 

2.1 

1.9 

Usefulness  of  telemedicine  technologies 

1.6 

1.5 

Perception  of  telemedicine/healthcare 

2.0 

1.7 

Note.  Lower  attitude  scores  indicate  more  positive  attitudes.  || 
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The  trend  extended  to  the  questions  from  which  each  attitude  variable  score  was 
derived.  Each  of  the  4  attitude  variables  consisted  of  5  to  7  different  questions. 
The  nonphysicians  showed  more  favorable  attitude  scores  on  the  majority  of 
questions  within  each  of  the  4  attitude  variables.For  example,  when  answering 
questions  about  their  satisfaction  with  different  technologies,  the  nonphysicians 
indicated  greater  satisfaction  than  the  physicians  on  5  of  the  7  technologies.  In 
particular,  the  nonphysicians  were  more  satisfied  than  the  physicians  on  the  RMS 
and  VTC.  On  the  other  hand,  the  physicians  were  more  satisfied  with  store-and- 
forward  and  email  than  the  nonphysicians. 

Responding  to  questions  about  their  satisfaction  with  specific  modes  of 
transmitted  information,  the  nonphysicians  rated  3  of  the  5  modes  higher  than  did 
the  physicians.  The  physicians’  ratings  for  these  modes  were  relatively  low, 
between  “somewhat  satisfied”  and  “undecided.”  The  differential  between  the 
physicians  and  nonphysicians  was  particularly  evident  in  the  reactions  to  sound 
and  video  images  transmitted  via  VTC.  Both  groups  were  undecided  on 
satisfaction  with  faxed  images.  Again,  the  physicians  were  more  satisfied  with 
emailed  images  and  store-and-forward  images  than  the  nonphysicians. 

In  their  answers  to  questions  about  the  usefulness  of  telemedicine  technologies, 
the  nonphysicians  rated  4  out  of  5  items  more  favorably  than  did  the  physicians. 
The  exception  was  that  the  physicians  rated  store-and-forward  imaging  more 
favorably  than  the  nonphysicians.  Both  groups,  however,  gave  high  ratings  on  the 
usefulness  of  VTC  (M=  1.4). 

Questioned  about  their  perceptions  of  telemedicine  and  healthcare,  the 
nonphysicians  gave  more  positive  responses  to  5  of  6  questions  than  did  the 
physicians.  The  physicians,  however,  indicated  a  stronger  belief  than  the 
nonphysicians  that  patients  were  satisfied  with  telemedicine  in  place  of  an  office 
visit.  Both  physicians  and  nonphysicians  supplied  equally  favorable  ratings  on  the 
cost  effectiveness  of  telemedicine  (M  =  1.4).  Both  groups  were  generally 
undecided  when  asked  whether  the  quality  of  healthcare  via  telemedicine  was 
inferior  to  in-person  care.  This  particular  question  received  the  least  favorable 
rating  of  all  items  in  the  survey. 

4.4  Comments  by  Medical  Personnel 

At  the  end  of  the  structured  question  section  of  the  survey,  the  medical  personnel 
were  asked  if  they  had  additional  comments.  Four  of  the  nonphysicians  and  three 
of  the  physicians  offered  comments. 

4.4.1  Comments  by  Nonphysicians 

One  nonphysician  commented  on  the  need  for  better  coordination  between 
services  on  telemedicine  and  healthcare.  In  addition  to  indicating  that  the  VTC 
and  RMS  had  a  very  positive  impact,  another  nonphysician  respondent  expressed 
a  desire  for  office  desktop  access  to  VTC.  Another  nonphysician  commented  that 
any  technology  that  might  reduce  or  make  current  paperwork  easier  would  be 
beneficial.  One  nonphysician  commented  that  archived  videotapes  of  patient  visits 
would  be  useful  for  later  reference. 
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4.4.2  Comments  by  Physicians 

Two  physicians  provided  favorable  comments  about  the  VTC  but  included 
negative  reactions  to  the  audio  quality.  Another  physician  expressed  concern 
about  VTC  use  for  child  psychiatry.  This  individual  believed  that  the  audio  and 
video  quality  of  the  current  VTC  system  was  too  poor  to  facilitate  the  accurate 
evaluation  of  children.  Therefore,  this  individual  indicated  that  VTC  evaluations 
for  child  psychiatry  had  been  discontinued. 

5.0  Discussion 

The  present  results  support  several  conclusions: 

•  The  ENT  consultations  had  substantial  clinical  impact.  Nearly  half  of  these  VTC 
consultations  led  to  a  change  in  diagnosis  by  the  ENT  specialist  at  NMCSD 
relative  to  the  initial  diagnosis  by  the  primary  provider  at  the  remote  MTF.  This 
finding  was  robust,  generalizing  across  different  types  of  ENT  conditions,  patient 
demographics,  and  MTFs. 

•  The  medical  personnel  in  TRICARE  Region  9  had  substantial  telemedicine 
experience  and  had  generally  positive  attitudes  toward  the  telecommunications 
technologies  and  the  telemedicine  process. 

•  The  nonphysicians  had  slightly  more  positive  attitudes  about  telemedicine  than 
did  the  physicians.  This  trend  was  interesting  but  nonsignificant,  given  the  small 
samples.  The  nonphysicians  also  used  telemedicine  more  and  in  more  ways  than 
did  the  physicians. 

5.1  Clinical  Impact 

Forty-five  percent  of  ENT  consultations  led  to  a  change  in  diagnosis.  This  finding 
validates  the  substantial  rate  of  diagnosis  change  (49%)  reported  in  the  initial 
study  of  TRICARE  Region  9  telemedicine  (Melcer  et  al.,  2001).  The  data  in  the 
previous  study  were  difficult  to  interpret  because  some  observations  were 
missing,  which  may  have  biased  the  observed  rate  of  clinical  impact.  The  present 
study  provided  a  complete  and  continuous  sample  of  ENT  consultations  during  a 
planned  4-month  period. 

This  clinical  finding  raises  further  issues: 

•  Since  the  effectiveness  of  consultations  extended  to  active-duty  personnel, 
will  present  telemedicine  procedures  be  cost-effective  aboard  ships?  This 
question  is  especially  pertinent  since  the  types  of  ENT  problems  in  this 
study  (allergies,  sleep-related)  could  influence  shipboard  operational 
efficiency.  Upper  respiratory  conditions  and  allergies  may  be  a  special 
concern  for  small  ships,  possibly  due  to  the  restricted  environments  on 
these  vessels  (Blood  &  Griffith,  1990).  Some  consultations  (e.g., 
tonsillitis-related)  could  determine  the  need  for  surgery  and  evacuation. 
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•  Will  the  evaluation  of  consultations  similar  to  those  in  this  study, 
conducted  by  other  ENT  specialists,  extend  the  generality  of  the  present 
findings? 

•  Why  did  some  consultations  lead  to  a  change  in  diagnosis  while  others  did 
not?  Further  analysis  combining  data  from  the  present  study  and  the 
previous  evaluation  would  allow  a  strong  test  of  the  predictors  of  clinical 
impact. 

•  Is  it  possible  to  decrease  the  rate  of  referrals  based  on  the  educational 
benefit  of  the  consultation  process  for  primary  providers?  With  increased 
exposure  to  specialty  consultations,  the  primary  providers  who  generate 
the  consults  (e.g.,  GMO,  IDC)  may  learn  to  recognize  difficult  diagnoses 
on  their  own.  This  would  provide  valuable  training  for  shipboard  medical 
duty,  where  the  availability  of  consultations  would  be  limited. 

5.2  Attitudes  and  Telemedicine  Experience 

The  interviews  with  medical  personnel  revealed  generally  favorable  attitudes 
toward  telemedicine  technologies  and  the  telemedicine  process.  In  other 
telemedicine  studies,  such  positive  findings  on  satisfaction  with  telemedicine  have 
been  frequently  reported.  It  has  been  suggested  that  this  is  because  patients  and 
providers  are  not  asked  critical  questions  (Whitten  &  Mair,  2000). 

The  present  questionnaire  instrument  was  designed  to  include  such  critical 
questions.  The  ratings  of  medical  personnel  were  less  favorable  in  response  to 
some  of  these  questions,  such  as  “Is  the  quality  of  care  rendered  through  the  use 
of  telemedicine  inferior  to  that  provided  in  person?”  Other  questions  that  drew 
less  favorable  responses  were  related  to  satisfaction  with  VTC  audio  and  video 
and  the  quality  of  images  transmitted  by  fax. 

Since  certain  critical  questions  did  produce  less  favorable  ratings,  the  overall 
positive  outcomes  appear  to  reflect  actual  satisfaction  with  telemedicine  rather 
than  the  bias  of  the  instrument.  Also,  the  trend  for  physicians  and  nonphysicians 
to  show  different  patterns  of  response  suggests  that  the  survey  instrument  has 
some  predictive  validity.  This  study  has  addressed  several  internal  validity  issues. 
The  present  results  are  supportive  of  the  reliability  and  validity  of  the  survey 
instrument.  Further  data  collection  would  strengthen  conclusions  as  to  whether 
these  measures  produce  an  unbiased  and  reliable  instrument. 

The  interviews  showed  that  medical  personnel,  particularly  physicians,  were 
concerned  that  telemedicine  might  compromise  patient  care  in  some  ways  relative 
to  in-person  treatment.  In  contrast,  most  personnel  agreed  with  statements  that 
telemedicine  optimized  overall  care  for  patients.  This  suggests  that  the  personnel 
are  willing  to  use  telemedicine  and  work  through  its  technical  pitfalls  to  achieve 
the  goal  of  providing  timely,  convenient,  and  accessible  healthcare. 

Perhaps,  medical  personnel  could  make  more  regular  use  of  telemedicine  for 
applications  other  than  diagnosis  confirmation.  The  availability  of  the  Internet  and 
other  resources  for  patient  and  provider  medical  reference  and  education  could  be 
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stressed.  For  instance,  the  Navy  information  center  currently  operates  a  “virtual 
hospital”  on-line  for  this  purpose  (http://www.vnh.org). 

The  use  of  VTC  for  ENT  care  in  TRICARE  Region  9  clearly  profited  from  a 
functional  infrastructure,  namely  the  telemedicine  coordinator,  medical  personnel, 
and  RMS  technology.  One  key  element  was  certainly  the  continuity  of  personnel, 
such  as  the  lead  ENT  specialist,  during  the  past  few  years  in  TRICARE  Region  9. 
This  continuity  provided  the  time  and  expertise  to  work  through  problems  with 
new  technology. 

In  some  telemedicine  specialties,  however,  such  as  child  psychiatry,  the  technical 
limitations  of  VTC  may  have  prevented  the  growth  of  a  telemedicine  specialty. 
Clearly,  specific  technical  needs  must  be  anticipated  for  each  telemedicine 
specialty  to  function  successfully. 

The  RMS  allowed  the  primary  provider  and  the  consultant  a  common  Internet- 
based  system  to  manage  and  record  telemedicine  activity.  Medical  personnel 
reported  relatively  high  satisfaction  with  the  RMS,  but  noted  the  need  to  “work 
out  some  bugs.”  TRICARE  Region  9  is  presently  installing  a  next  generation 
system  to  replace  the  RMS. 

5.3  Physician  and  Nonphysician  Telemedicine  Profiles 

Analysis  of  interviews  contrasted  physician  and  nonphysician  medical  personnel. 
A  small  but  consistent  pattern  of  differences  between  these  two  groups  emerged. 

Nonphysicians  reported  a  higher  rate  of  telemedicine  use  than  the  physicians 
during  the  present  study  period.  This  higher  rate  was  due  to  the  fact  that  the 
nonphysicians  often  presented  patients  and  executed  the  telemedicine  sessions  for 
a  referring  provider. 

Could  this  higher  rate  of  telemedicine  use  help  to  explain  the  nonphysicians’  more 
favorable  attitudes  toward  telemedicine?  Experience  and  exposure  to  telemedicine 
have  been  shown  to  be  directly  related  to  use  and  positive  attitudes  (Karp  et  al., 
2000;  Walters,  1996).  However,  the  overall  correlation  between  technology 
experience  and  telemedicine  attitudes  in  this  study  was  calculated  and  was  not 
significant.  This  finding  is  presented  here  tentatively  due  to  the  small  sample  size. 

In  any  case,  it  did  appear  that  nonphysicians  benefited  from  the  use  of 
telemedicine,  which  provided  a  means  to  resolve  specialized  patient  healthcare 
problems.  In  contrast,  the  physicians’  responses  seemed  more  skeptical.  This  may 
reflect  their  role  as  the  ultimate  patient  care  decision-makers,  which  may  increase 
their  focus  on  the  limits  of  telemedicine.  The  physicians  may  also  view 
telemedicine  less  favorably  because  it  presents  technical  problems,  which  they 
lack  the  time  or  resources  to  resolve. 

The  physicians  appeared  to  prefer  email  and  store-and-forward  technologies  more 
than  the  nonphysicians.  Physicians  may  prefer  the  convenience  of  stored  records 
to  review  in  detail  as  their  schedules  allow.  It  is  also  possible  that  they  view  VTC 
as  unreliable.  Given  that  VTC  can  be  expensive  and  technically  difficult,  it  seems 
important  to  determine  how  it  enhances  patient  care  relative  to  store-and-forward. 
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Medical  background  and  the  attitudes  of  providers  have  been  related  to  the  use 
and  clinical  impact  of  telemedicine  (Karp  et  al.,  2000;  Nice,  1987).  One  important 
issue  for  naval  applications  is  the  contrast  between  physician  and  nonphysician 
medical  personnel  (Nice,  1987).  Nonphysicians  such  as  corpsmen  provide 
extensive  medical  support  for  physicians  (e.g.,  GMOs)  in  large  shipboard  medical 
departments  (e.g.,  carriers,  amphibious).  Corpsmen  provide  all  medical  care 
aboard  small  ships  (cruisers,  destroyers)  with  little  support  from  technology  or 
additional  medical  personnel.  Corpsmen  must  function  autonomously  at  sea.  The 
positive  attitudes  of  nonphysicians  on  telemedicine  and  technology  would  likely 
enhance  the  success  of  any  small  ship  telemedicine  system. 

Recommendations  and  Future  Work 

•  Consider  cost-effectiveness  of  ENT  telemedicine  for  shipboard  medical 
departments.  Store-and-forward  video  with  standardized  exam  protocols 
would  eliminate  need  for  live  VTC  bandwidth  requirements  (e.g..  Stem  et  al., 
1998). 

•  Analyze  the  relationship  between  provider  training  (e.g.,  IDC  vs.  GMO)  and 
clinical  impact  of  ENT  consultations.  The  Lemoore  facility  offers  an 
opportunity  to  answer  this  question  because  it  has  high  volume  and  many 
referring  providers  to  a  central  point  of  VTC. 

•  Describe  MTF  capabilities  and  relate  them  to  telemedicine  use  and 
clinical  impact. 

•  The  collection  of  more  data  from  other  TRICARE  regions  would  increase  the 
sample  size  and  allow  two-tailed  t  tests  to  be  mn  to  determine  if  significant 
differences  in  telemedicine  attitudes  exist  between  physician  and 
nonphysician  groups.  This  data  collection  and  analysis  would  improve  the 
generalizability  of  the  results  to  other  populations. 

•  Explore  technologies  and  protocols  for  VTC  child  psychiatry.  Telepsychiatry 
has  gained  popularity  for  adult  patients  and  in  some  studies  of  child  psychiatry 
(Capner,  2000). 
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